Recorded data processing method and recorded data processing device

ABSTRACT

A recorded data processing method includes calculating, for each of N (N is a natural number of three or more) recorded data pairs each formed by two pieces of recorded data that are adjacent to each other when N pieces of recorded data each representing a recording target including at least one of audio or an image are arranged cyclically, a time difference between time signals representing temporal changes of the recording target in the two respective pieces of recorded data of the recorded data pair. It is also evaluated whether or not the N pieces of recorded data include recorded data having no relation according to a total value of N time differences calculated for the N recorded data pairs.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part application of InternationalApplication No. PCT/JP2017/008272, filed Mar. 2, 2017, which claimspriority to Japanese Patent Application No. 2016-045132 filed in Japanon Mar. 9, 2016. The entire disclosures of International Application No.PCT/JP2017/008272 and Japanese Patent Application No. 2016-045132 arehereby incorporated herein by reference.

BACKGROUND

The present technology relates to a processing recorded data.

Various technologies have been proposed for processing mutual temporalrelation between a plurality of pieces of recorded data including videoand audio. For example, Japanese Laid-Open Patent Application No.2008-193561 (hereinafter referred to as Patent Document 1) disclosestechnologies for analyzing a plurality of pieces of audio data eachrecorded at a time of capturing of a plurality of video obtained bycapturing a same subject from different positions, and thereby generatestime difference information for synchronizing the plurality of videos.Specifically, the time difference information is generated according toa shift time that maximizes a cross-correlation function between twopieces of audio data.

SUMMARY

A recorded data processing method in accordance with some embodimentsincluding calculating, for each of N (N is a natural number of three ormore) recorded data pairs each formed by two pieces of recorded datathat are adjacent to each other when N pieces of recorded data eachrepresenting a recording target including at least one of audio or videoare arranged cyclically, a time difference between time signalsrepresenting temporal changes of the recording target in the tworespective pieces of recorded data of the recorded data pair.

In accordance with an embodiment, the recorded data processing methodfurther includes evaluating whether or not the recorded data having norelevance among N pieces of recorded data according to a total value ofN time differences calculated for the N recorded data pairs.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a recorded data editing system according toa first embodiment.

FIG. 2 is a view illustrating N pieces of recorded data having relationto each other.

FIG. 3 is a view illustrating N pieces of recorded data including N−1pieces of recorded data having relation to each other and one piece ofrecorded data having relation to the N−1 pieces of recorded data.

FIG. 4 is a view illustrating N pieces of recorded data including N−1pieces of recorded data having relation to each other and one piece ofrecorded data having no relation to the N−1 pieces of recorded data.

FIG. 5 is a flowchart of a process for generating content by anelectronic controller.

FIG. 6 is a block diagram of a recorded data editing system according toa second embodiment.

FIG. 7 is a view illustrating of processing of moving evaluation targetrecorded data in an evaluating unit.

FIG. 8 is a flowchart of a process for generating content by anelectronic controller.

FIG. 9 is a view illustrating a case where the relation of recorded dataincluded in an unrelated data group is affirmed by reevaluation.

DESCRIPTION OF EMBODIMENTS

In conventional systems, there is a possibility that recording data mayinclude some non-related recording data and some related recording data.In the patent document 1, some shift times are calculated even when therecorded data to be analyzed include non-related data. In view of theabove circumstances, it is an object of some embodiments to evaluateappropriately whether or not the recorded data has any relevance among aplurality of recorded data.

Selected embodiments will now be explained with reference to thedrawings. It will be apparent to those skilled in the audio field fromthis disclosure that the following descriptions of the embodiments areprovided for illustration only and not for the purpose of limiting theinvention as defined by the appended claims and their equivalents.

First Embodiment

FIG. 1 is a block diagram of a recorded data editing system 10 accordingto a first embodiment. The recorded data editing system 10 is a computersystem for processing audio (for example, voice or musical audio) andvideo. As illustrated in FIG. 1, the recorded data editing system 10includes an electronic controller 22, a storage device 24, a recordeddata obtaining device 26, a display device 32, a audio emitting device34, and an operating device 36. In some embodiments, the recorded dataediting system 10 can be suitably implemented by a portable informationprocessing device such, for example, as a cellular phone, a smart phone,a tablet terminal, a personal computer, or the like. However, therecorded data editing system 10 can also be implemented by a stationaryinformation processing device.

The term “electronic controller” as used herein refers to hardware thatexecutes software programs.

The electronic controller 22 includes processing device (for example, acentral processing unit (CPU)) that controls the elements of therecorded data editing system 10. The recorded data obtaining device 26obtains recorded data X1 to XN resulting from capturing by N recordingdevices, individually. The recording devices are video apparatusesincluding a audio collecting device collecting audio and a video devicecollecting video by capturing video. Movie data includes audio data andvideo data. The recorded data is audio data or movie data. The recordingdevices are, for example, video apparatuses such as digital camcordersor the like and information terminals such as mobile telephones, smartphones, or the like having a recording function. Recorded data X is datarepresenting audio or movie (audio and video) recorded by a recordingdevice.

The recorded data obtaining device 26 in the first embodiment obtains Npieces of recorded data X1 to XN (N is a natural number of three ormore). Specifically, the recorded data obtaining device 26 can obtainthe N pieces of recorded data X1 to XN from the respective recordingdevices by publicly known short-range radio communication such, forexample, as wireless fidelity (Wi-Fi) (registered trademark), Bluetooth(registered trademark), or the like, or obtain the recorded data Xrecorded by each of the recording devices from a storage device storingthe recorded data X via a communication network such as the Internet orthe like.

The storage device 24 is formed by recording medium such, for example,as a magnetic recording medium, a semiconductor recording medium, or thelike. The storage device 24 stores a program executed by the electroniccontroller 22 and various kinds of data used by the electroniccontroller 22. The storage device 24 in the first embodiment stores theN pieces of recorded data X1 to XN obtained by the recorded dataobtaining device 26. It is also possible to store the N pieces ofrecorded data X1 to XN in the storage device 24 in advance. In thiscase, the recorded data obtaining device 26 can be omitted from therecorded data editing system 10. In addition, it is possible to installthe storage device 24 in a server with which the recorded data editingsystem 10 can communicate (that is, a cloud storage). In this case, thestorage device 24 can be omitted from the recorded data editing system10.

The plurality of recording devices, for example, record, in parallelwith each other, audio and video as a common recording target (recordingobject) at mutually different positions. For example, the plurality ofrecording devices are arranged at mutually different positions in acommon acoustic space such as a hall, a concert hall, a dance hall orthe like, and each of the plurality of recording devices generates therecorded data X by recording a state of a stage and audience, forexample, from a different angle. The recorded data X in the firstembodiment represents the recording target including the audio collectedby an audio collecting device and the video collected by an videodevice. Specifically, the recorded data X includes a audio signalrepresenting temporal changes in the audio collected by the audiocollecting device and a video signal representing temporal changescaptured by the video device (that is, a movie). For example, in a casewhere the played audio of a musical piece for a play performed on astage is reproduced from a audio emitting device (for example, aspeaker) installed on the stage, the audio of the recorded data Xrecorded by each of the recording devices commonly includes the playedaudio (though audio characteristics such as volume and the like candiffer). Incidentally, while the recording target including both theaudio and the image is illustrated, a recording target including onlyone of audio and an image can also be assumed.

A user of each recording device separately starts recording by the ownrecording device. Hence, a start point of recording of the audio and themoving image does not precisely coincide between the N pieces ofrecorded data X1 to XN, but can differ for each piece of recorded dataX. That is, there are time differences between the N pieces of recordeddata X1 to XN.

The storage device 24 basically stores the N pieces of recorded data X1to XN having relation to each other. However, there is a possibilitythat due to a mistake in operation of a user of a recording device orthe like, for example, recorded data X having no relation to the otherrecorded data X may be mixed in the N pieces of recorded data X1 to XNstored in the storage device 24. The pieces of recorded data X havingrelation to each other, for example, have relation such that one of orboth a subject and a location are mutually common or related (relationin terms of content) and recording periods of the pieces of recordeddata X partly overlap each other on a time axis (relation in terms oftime). For example, relation is ensured between N−1 pieces of recordeddata X1 to XN−1 recorded by reliable users (for example, personsconcerned in an event) so as to be related to each other in terms ofcontent and time. In addition, when a case is assumed in which dataindicating recording positions or recording times is added to therecorded data X, N−1 pieces of recorded data X1 to XN−1 having relationto each other can be secured by analyzing commonality between therecording positions or the recording times. On the other hand, recordeddata XN having no relation is recorded data X that does not have atleast one of relation in terms of content and relation in terms of time.The recorded data XN having no relation is, for example, recorded data Xobtained by performing recording in a same period as the recordingperiods of the other recorded data X1 to XN−1 but recording a state of adifferent event (in a case of a different subject), or recorded data Xobtained by recording the same event but starting the recording after anend of recording for the other recorded data X1 to XN−1 (in a case wherethe recording period does not partially overlap on the time axis). Thatis, the recorded data XN having no relation does not include commonaudio included in the other plurality of pieces of recorded data X1 toXN−1. The first embodiment assumes a case where the N pieces of recordeddata X1 to XN include the N−1 pieces of recorded data X1 to XN−1 havingrelation to each other and one piece of recorded data XN having unknownrelation to the N−1 pieces of recorded data X1 to XN−1. The N−1 piecesof recorded data X1 to XN−1 are recorded data X whose mutual relation isconfirmed in advance by an arbitrary method.

The display device 32 (for example, a liquid crystal display panel) inFIG. 1 displays an image specified from the electronic controller 22.The audio emitting device 34 (for example, a speaker or headphones)emits audio specified from the electronic controller 22. The operatingdevice 36 is an input apparatus receiving an instruction from a user.The operating device 36 is, for example, formed by a plurality ofoperating elements detecting operations by the user or a touch paneldetecting contact of the user with a display surface of the displaydevice 32.

The electronic controller 22 implements a plurality of functions (arecorded data analyzing unit 40 and an edit processing unit 46) forprocessing the N pieces of recorded data X1 to XN by executing theprogram stored in the storage device 24. Incidentally, it is alsopossible to adopt a configuration in which some of the functions of theelectronic controller 22 are implemented by an electronic circuitdedicated to audio processing or image processing or a configuration inwhich the functions of the electronic controller 22 are distributed to aplurality of devices.

As illustrated in FIG. 2, the recorded data analyzing unit 40 determinesa time difference Oij (i, j=1 to N, i≠j) between two pieces of recordeddata Xi and Xj that are adjacent to each other when the N pieces ofrecorded data X1 to XN generated by the recording devices are arrangedcyclically, and evaluates whether or not the N pieces of recorded dataX1 to XN include the recorded data XN having no relation from a totalvalue S of time differences Oij. The cyclic arrangement of the N piecesof recorded data X1 to XN means an arrangement (annular arrangement) inwhich the N pieces of recorded data X1 to XN are arranged in series witheach other and the first recorded data X1 is made to follow the lastrecorded data XN. Hence, the cyclic arrangement of the N pieces ofrecorded data X1 to XN includes a pair (hereinafter referred to as a“recorded data pair”) Pij formed by two pieces of recorded data Xi andXj adjacent to each other. That is, there are N combinations between thenumerical value i and the numerical value j: (i, j)=(1, 2), (2, 3), . .. , (N−1, N), (N, 1). That is, the N pieces of recorded data include Nrecorded data pairs P12 to PN·1. As is understood from FIG. 2, the timedifference Oij means a relative time (offset) of the recorded data Xjwhen the recorded data Xi is set as a reference. Incidentally, thepermutation of the N pieces of recorded data X1 to XN arrangedcyclically is arbitrary.

As illustrated in FIG. 1, the recorded data analyzing unit 40 in thefirst embodiment includes a calculating unit 42 and an evaluating unit44. For each of the N recorded data pairs P12 to PN·1 each formed by twopieces of recorded data Xi and Xj that are adjacent to each other whenthe N pieces of recorded data X1 to XN are arranged cyclically, thecalculating unit 42 calculates a time difference Oij between audiosignals (an example of time signals) in the two respective pieces ofrecorded data Xi and Xj of the recorded data pair Pij.

Specifically, for each of the N recorded data pairs P12 to PN·1, thecalculating unit 42 calculates a time difference τ maximizing anabsolute value |Cij(τ)| of a audio signal cross-correlation Cij(τ)between the recorded data Xi and the recorded data Xj as the timedifference Oij of the recorded data pair Pij. As expressed by thefollowing Equation (1), the cross-correlation Cij(τ) is a numericalstring indicating a degree of time waveform correlation between a audiosignal yi(t) included in the recorded data Xi and a audio signal yj(t)included in the recorded data Xj with a time difference (amount of shifton the time axis) τ of the audio signal yj(t) with respect to the audiosignal yi(t) as a variable after a starting point of the audio signalyi(t) and a starting point of the audio signal yj(t) are made tocoincide with each other on the time axis. In a case where the recordeddata Xi and the recorded data Xj have no relation to each other, therecorded data Xi and the recorded data Xj do not include common audio,as described earlier, and therefore the cross-correlation Cij(τ) doesnot assume a significant value. Incidentally, the time difference τ canassume a negative numerical value. Hence, for example, when the recordeddata Xj is positioned in the rear of the recorded data Xi on the timeaxis, the time difference Oij is a positive number, and when therecorded data Xj is positioned in front of the recorded data Xi on thetime axis, the time difference Oij is a negative number.

$\begin{matrix}\left\lbrack {{Expression}\mspace{14mu} 1} \right\rbrack & \; \\{{C_{ij}(\tau)} = {\frac{1}{N + 1}{\sum\limits_{t = 0}^{N}{{y_{i}(t)}{y_{j}\left( {t + \tau} \right)}}}}} & (1)\end{matrix}$

In addition, as expressed by Equation (2), the cross-correlation Cij(τ)can also be calculated by an inverse Fourier transform (IFFT) of a crossspectrum of a frequency spectrum Yi(f) of the audio signal yi(t) and afrequency spectrum Yj(f) of the audio signal yj(t). f denotes frequency.Yi*(f) is a complex conjugate of Yi(f). A configuration that calculatesthe cross-correlation Cij(τ) by operation of Equation (2) has anadvantage of being able to reduce an amount of calculation incalculating the cross-correlation Cij(τ) as compared with aconfiguration that calculates Equation (1).[Expression 2]C _(ij)(τ)=IFFT(Y _(i)*(f)Y _(j)(f))  (2)

FIG. 3 is a view illustrating the N−1 pieces of recorded data X1 to XN−1having relation to each other. As illustrated in FIG. 2, when the N−1pieces of recorded data X1 to XN−1 have relation to each other, a totalvalue S (S=O12+O23+ . . . +ON−1·1) of N−1 time differences O12 to ON−1·1calculated for the N−1 recorded data pairs P12 to PN−1·1, respectively,is a numerical value close to zero (is ideally zero).

Here, a case is assumed in which the N pieces of recorded data X1 to XNare provided by adding, to the N−1 pieces of recorded data X1 to XN−1having relation to each other, one piece of recorded data XN havingunknown relation thereto (which recorded data will hereinafter bereferred to as “unknown recorded data”). When the unknown recorded dataXN has relation to the recorded data X1 to XN−1, as illustrated in FIG.2, the total value S (S=O12+O23+ . . . +ON·1) of time differences O12 toON·1 calculated for the N recorded data pairs P12 to PN·1, respectively,is a numerical value close to zero (is ideally zero). When the unknownrecorded data XN has no relation to the recorded data X1 to XN−1, on theother hand, as illustrated in FIG. 4, the recorded data XN−1 and theunknown recorded data XN do not partly overlap each other on the timeaxis (that is, the recorded data XN−1 and the unknown recorded data XNhave no relation to each other), and therefore the cross-correlationCN−1·N(τ) of the recorded data pair PN−1·N is not a significant value.Hence, the total value S of the N time differences O12 to ON·1 is anumerical value deviating from zero. As is understood from the abovedescription, the total value S of the time differences O12 to ON·1 canbe used as an index indicating whether or not the recorded data XNhaving no relation is included in the N pieces of recorded data X1 to XN(whether or not the unknown recorded data X has relation to the recordeddata X1 to XN−1). That is, when the total value S of the N timedifferences O12 to ON·1 deviates from zero, it can be estimated that theN pieces of recorded data X1 to XN include the recorded data XN havingno relation.

In consideration of the above tendency, the evaluating unit 44 in FIG. 1evaluates whether or not the N pieces of recorded data X1 to XN includethe recorded data XN having no relation according to the total value Sof the N time differences O12 to ON·1 calculated for the N recorded datapairs P12 to PN·1, respectively. In the first embodiment, as describedearlier, the N pieces of recorded data X1 to XN include the N−1 piecesof recorded data X1 to XN−1 having relation to each other and the onepiece of unknown recorded data XN having unknown relation to the N−1pieces of recorded data X1 to XN−1. Under the above assumption, when theN pieces of recorded data X1 to XN include the recorded data XN havingno relation, it means that the unknown recorded data XN has no relationto the other N−1 pieces of recorded data X1 to XN−1. Incidentally, whilethe unknown recorded data XN is added to an end of the arrangement ofthe N−1 pieces of recorded data X1 to XN−1 in the first embodiment, theposition at which the unknown recorded data XN is added is arbitrary.

When the total value S of the time differences O12 to ON·1 calculatedfor the N recorded data pairs P12 to PN·1, respectively, exceeds apredetermined threshold value, the evaluating unit 44 in the firstembodiment determines that the unknown recorded data XN is the recordeddata XN having no relation to the N−1 pieces of recorded data X1 toXN−1, and excludes the unknown recorded data XN from the N pieces ofrecorded data X1 to XN. When the total value S of the N time differencesO12 to ON·1 calculated for the N recorded data pairs P12 to PN·1,respectively, is less than the predetermined threshold value, on theother hand, the evaluating unit 44 in the first embodiment determinesthat the unknown recorded data XN is the recorded data XN havingrelation to the N−1 pieces of recorded data X1 to XN−1, and does notexclude the unknown recorded data XN from the N pieces of recorded dataX1 to XN. Incidentally, the threshold value is selected empirically orstatistically.

When the evaluating unit 44 determines that the unknown recorded data XNhas no relation to the N−1 pieces of recorded data X1 to XN−1, the editprocessing unit 46 in FIG. 1 generates content Z in which the N−1 piecesof recorded data X1 to XN−1 are synchronized with each other. When theevaluating unit 44 determines that the unknown recorded data XN hasrelation to the N−1 pieces of recorded data X1 to XN−1, on the otherhand, the edit processing unit 46 generates content Z in which the Npieces of recorded data X1 to XN are synchronized with each other. Thesynchronization of the recorded data X means a state in which the timeaxes of audio and moving images of the respective pieces of recordeddata X are made to coincide with each other in the plurality of piecesof recorded data X. Incidentally, while a publicly known technology canbe arbitrarily adopted for the synchronization of the recorded data X,it is also possible to synchronize the plurality of pieces of recordeddata X by using the time differences Oij calculated by the calculatingunit 42. The content Z generated by the edit processing unit 46 in FIG.1 are reproduced according to an instruction of the electroniccontroller 22. Specifically, the moving image of the content Z isdisplayed by the display device 32, and the audio of the content Z isemitted by the audio emitting device 34.

Incidentally, according to a configuration that synchronizes the Npieces of recorded data X1 to XN without excluding the recorded data XNhaving no relation to the N−1 pieces of recorded data X1 to XN−1, therelation on the time axis of the recorded data XN that is not supposedto be synchronized is estimated erroneously. In the first embodiment,the recorded data XN having no relation to the N−1 pieces of recordeddata X1 to XN−1 is excluded, and therefore only the pieces of recordeddata X having relation to each other can be synchronized with eachother.

Incidentally, also assumed as a method of excluding the recorded data XNhaving no relation to the N−1 pieces of recorded data X1 to XN−1 fromthe N pieces of recorded data X1 to XN is a configuration (hereinafterreferred to as a “contrast example”) that, for example, refers toinformation indicating the recording periods (for example, start timesand end times) of the recorded data X, and excludes the recorded data XNhaving no relation in terms of time to the N−1 pieces of recorded dataX1 to XN−1. However, in the contrast example, the recorded data XNhaving relation in terms of time as a result of being recorded in a sameperiod but having no relation in terms of content is erroneously decidedto have relation to the N−1 pieces of recorded data X1 to XN−1, and isconsequently not excluded. On the other hand, the first embodiment candetermines that the recorded data XN has no relation to the N−1 piecesof recorded data X1 to XN−1 according to the total value S of the timedifferences Oij, and can therefore appropriately exclude the recordeddata XN having relation in terms of time but having no relation in termsof content. In addition, there is an advantage of being able to evaluaterecorded data X having no information indicating a recording period.

FIG. 5 is a flowchart of a process for generating content Z by theelectronic controller 22 generates content Z. The processing of FIG. 5is started by being triggered by an instruction from a user to theoperating device 36. As an example, a case is assumed in which therecorded data obtaining device 26 obtains N pieces of recorded data X1to XN including N−1 pieces of recorded data X1 to XN−1 having relationto each other, the N−1 pieces of recorded data X1 to XN−1 having beenobtained by recording a state of a play performed on a stage frommutually different positions, and one piece of unknown recorded data Xhaving unknown relation to the N−1 pieces of recorded data X1 to XN−1.When the processing of FIG. 5 is started, the calculating unit 42calculates, for each of N recorded data pairs PP12 to PN·1, a timedifference τ maximizing the absolute value |Cij(τ)| of thecross-correlation Cij(τ) as a time difference Oij (O12 to ON·1) betweenthe audio signal yi(t) and the audio signal yj(t) of the recorded datapair Pij (SA1).

The evaluating unit 44 calculates a total value S of the N timedifferences O12 to ON·1 calculated for the N recorded data pairs P12 toPN·1, respectively (SB1). Next, the evaluating unit 44 evaluates whetheror not the N pieces of recorded data X1 to XN include the recorded dataXN having no relation by comparing the total value S and a thresholdvalue with each other (SB2). Specifically, when the total value S of theN time differences O12 to ON·1 calculated for the N recorded data pairsP12 to PN·1, respectively, exceeds a predetermined threshold value, theevaluating unit 44 determines that the unknown recorded data X is therecorded data XN having no relation to the N−1 pieces of recorded dataX1 to XN−1 (SB2: YES), and excludes the unknown recorded data X from theN pieces of recorded data X1 to XN (SB3). When the total value S of theN time differences O12 to ON·1 calculated for the N recorded data pairsP12 to PN·1, respectively, is less than the predetermined thresholdvalue, on the other hand, the evaluating unit 44 determines that theunknown recorded data X has relation to the N−1 pieces of recorded dataX1 to XN−1 (SB2: NO), and does not exclude the unknown recorded data Xfrom the N pieces of recorded data X1 to XN (SB4).

The edit processing unit 46 generates content Z on the basis of a resultof the processing by the evaluating unit 44 (SC1). Specifically, whenthe evaluating unit 44 excludes the recorded data XN having no relation(SB3), the edit processing unit 46 generates the content Z in which theN−1 pieces of recorded data X1 to XN−1 are synchronized with each other.When the evaluating unit 44 does not exclude the unknown recorded dataXN from the N pieces of recorded data X1 to XN (SB4), the editprocessing unit 46 generates the content Z in which the N pieces ofrecorded data X1 to XN including the unknown recorded data XN aresynchronized with each other. As described above, the calculating unit42 performs the processing (step SA1) of calculating the N timedifferences O12 to ON·1 of the N recorded data pairs P12 to PN·1, andthe evaluating unit 44 performs the processing (steps SB1 to SB4) ofevaluating whether or not the N pieces of recorded data X1 to XN includethe recorded data XN having no relation.

As is understood from the above description, the first embodiment canappropriately evaluate whether or not the N pieces of recorded data X1to XN include the recorded data XN having no relation according to thetotal value S of the N time differences O12 to ON·1 respectivelycalculated for the N recorded data pairs P12 to PN·1 when the N piecesof recorded data X1 to XN are arranged cyclically.

Merely evaluating the presence or absence of recorded data X having norelation according to the total value S of the N time differences O12 toON·1 respectively calculated for N pieces of recorded data X1 to XN thatare unknown as to whether or not there is relation therebetween cannotidentify recorded data X having no relation to the other recorded data Xamong the N pieces of recorded data X. The first embodiment evaluatesthe presence or absence of relation according to the total value S ofthe N time differences O12 to ON·1 of the N recorded data pairs P12 toPN·1, particularly for the N pieces of recorded data X1 to XN includingthe N−1 pieces of recorded data X1 to XN−1 already known to haverelation to each other and one piece of recorded data X having unknownrelation to the N−1 pieces of recorded data X1 to XN−1. The firstembodiment can therefore determine whether or not the one piece ofrecorded data XN has relation to the N−1 pieces of recorded data X1 toXN−1.

Second Embodiment

The second embodiment will be described. In each of the embodimentsillustrated in the following, elements that have the same actions orfunctions as in the first embodiment have been assigned the samereference symbols as those used to describe the first embodiment, andthe detailed descriptions thereof have been appropriately omitted.

Whereas the first embodiment evaluates whether or not one piece ofunknown recorded data X has relation to the other recorded data X, thesecond embodiment evaluates whether or not each of a plurality of piecesof recorded data X has relation to other recorded data X.

FIG. 6 is a block diagram of a recorded data editing system 10 accordingto the second embodiment. Whereas the storage device 24 in the firstembodiment stores N−1 pieces of recorded data X1 to XN−1 having relationto each other and one piece of unknown recorded data XN, a storagedevice 24 in the second embodiment stores a plurality of pieces ofrecorded data X related to each other and a plurality of pieces ofunknown recorded data X having unknown relation to the other recordeddata X.

As illustrated in FIG. 7, processing of a recorded data analyzing unit40 in the second embodiment assumes, for convenience, a set of aplurality of pieces of recorded data X already known to have relation toeach other (which set will hereinafter be referred to as a “related datagroup”) C1, a set of a plurality of pieces of unknown recorded data Xthat are unknown as to whether or not the plurality of pieces of unknownrecorded data X have relation to the recorded data X within the relateddata group C1 (which set will hereinafter be referred to as an “unknowndata group”) C2, and a set of recorded data X whose relation to therecorded data X in the related data group C1 is negated (which set willhereinafter be referred to as an “unrelated data group”) C3 among theplurality of pieces of unknown recorded data X in the unknown data groupC2. As illustrated in FIG. 6, the recorded data analyzing unit 40 in thesecond embodiment has a configuration obtained by adding a selectingunit 48 to a calculating unit 42 and an evaluating unit 44 similar tothose of the first embodiment. The recorded data analyzing unit 40 inthe second embodiment evaluates presence or absence of relation of eachof the plurality of pieces of unknown recorded data X included in theunknown data group C2 to the plurality of pieces of recorded data X inthe related data group C1.

The selecting unit 48 in FIG. 6 sequentially selects one piece ofunknown recorded data X as evaluation target recorded data XN from theunknown data group C2. As in the first embodiment, the calculating unit42 in the second embodiment calculates a time difference Oij for eachrecorded data pair Pij of N pieces of recorded data X1 to XN includingthe one piece of evaluation target recorded data XN selected by theselecting unit 48 and the plurality (N−1) of pieces of recorded data X1to XN−1 within the related data group C1. The evaluating unit 44 in thesecond embodiment evaluates, as in the first embodiment, presence orabsence of relation of the evaluation target recorded data XN to each ofthe pieces of recorded data X within the related data group C1 (whetheror not the recorded data XN having no relation is included in the Npieces of recorded data X1 to XN) according to a total value S of N timedifferences O12 to ON·1. As illustrated in FIG. 7, when the relation ofthe evaluation target recorded data XN is affirmed, the evaluating unit44 in the second embodiment moves the evaluation target recorded data XNfrom the unknown data group C2 to the related data group C1. When therelation of the evaluation target recorded data XN is negated, on theother hand, the evaluating unit 44 in the second embodiment moves theevaluation target recorded data XN from the unknown data group C2 to theunrelated data group C3. The processing in the recorded data analyzingunit 40 (the selecting unit 48, the calculating unit 42, and theevaluating unit 44) is repeated until the unknown data group C2 iscleared of the recorded data X, that is, until the unknown data group C2becomes empty. After the processing in the recorded data analyzing unit40 is completed, an edit processing unit 46 in the second embodimentgenerates, as in the first embodiment, content Z in which the pluralityof pieces of recorded data X included in the related data group C1 aresynchronized with each other.

FIG. 8 is a flowchart of a process for generating content Z by theelectronic controller 22 in the second embodiment. As in the firstembodiment, the processing of FIG. 8 is started by being triggered by aninstruction from a user to an operating device 36. When the processingof FIG. 8 is started, the selecting unit 48 selects one piece ofevaluation target recorded data XN from the unknown data group C2 (SD1).The calculating unit 42 calculates a time difference Oij for eachrecorded data pair Pij of the N pieces of recorded data X1 to XNincluding the one piece of evaluation target recorded data XN selectedby the selecting unit 48 and the plurality (N−1) of pieces of recordeddata X1 to XN−1 within the related data group C1 (SA1). The evaluatingunit 44 calculates a total value S of N time differences O12 to ON·1calculated for N recorded data pairs P12 to PN·1, respectively (SB1).Next, the evaluating unit 44 evaluates whether or not the N pieces ofrecorded data X1 to XN include the recorded data XN having no relationby comparing the total value S and a threshold value with each other(SB2). Specifically, when the total value S of the N time differencesO12 to ON·1 calculated for the N recorded data pairs P12 to PN·1,respectively, exceeds a predetermined threshold value, the evaluatingunit 44 determines that the evaluation target recorded data XN is therecorded data XN having no relation to the related data group C1 (SB2:YES), and moves the evaluation target recorded data XN from the unknowndata group C2 to the unrelated data group C3 (SB3). When the total valueS of the N time differences O12 to ON·1 calculated for the N recordeddata pairs P12 to PN·1, respectively, is less than the predeterminedthreshold value, on the other hand, the evaluating unit 44 determinesthat the evaluation target recorded data XN is the recorded data XNhaving relation to the related data group C1 (SB2: NO), and moves theevaluation target recorded data XN from the unknown data group C2 to therelated data group C1 (SB4). Hence, a total number of pieces of recordeddata X in the related data group C1 is increased by one. Then, when theevaluating unit 44 moves the evaluation target recorded data XN to therelated data group C1 (that is, when the relation of the evaluationtarget recorded data XN to the related data group C1 is affirmed), theevaluating unit 44 moves all of the recorded data X included in theunrelated data group C3 to the unknown data group C2 (SB5). That is, theunrelated data group C3 is initialized.

Here, in a case where one piece of evaluation target recorded data XN ismoved from the unknown data group C2 to the related data group C1, whenthe presence or absence of relation of recorded data X within theunrelated data group C3 whose relation is negated in evaluation in thepast is evaluated again, the relation may be affirmed. For example, asillustrated in FIG. 9, when relation is evaluated between three piecesof recorded data X, that is, recorded data X1 and recorded data X2included in the related data group C1 and evaluation target recordeddata XN (broken line), the evaluation target recorded data XN does notpartly overlap the recorded data X2 on the time axis, and therefore theevaluation target recorded data XN is decided to have no relation to therelated data group C1 (recorded data X1 and X2) and is added to theunrelated data group C3. However, when the evaluating unit 44 addsrecorded data X3 to the related data group C1, the evaluation targetrecorded data XN (solid line) included in the unrelated data group C3partly overlaps the recorded data X3 on the time axis, as illustrated inFIG. 9. Thus, when the evaluating unit 44 performs evaluation again, therelation of the evaluation target recorded data XN to the related datagroup C1 (recorded data X1 to X3) is affirmed. Accordingly, when theevaluating unit 44 moves the evaluation target recorded data XN from theunknown data group C2 to the related data group C1 (SB4), the evaluatingunit 44 moves all of the recorded data X included in the unrelated datagroup C3 to the unknown data group C2 (SB5), as described above, and theevaluating unit 44 performs reevaluation.

After the evaluating unit 44 moves the evaluation target recorded dataXN (SB3 and SB4), the selecting unit 48 determines whether or not thereis recorded data X in the unknown data group C2, that is, whether or notthe unknown data group C2 is empty (SD2). When the unknown data group C2is empty (SD2: YES), the edit processing unit 46 generates content Z inwhich the plurality of pieces of recorded data X included in the relateddata group C1 are synchronized with each other (SC1). When the unknowndata group C2 is not empty (SD2: NO), on the other hand, the recordeddata analyzing unit 40 repeats the processing of steps SD1 to SB2 foreach piece of unknown recorded data X within the unknown data group C2.As described above, the selecting unit 48 performs the processing (SD1and SD2) of selecting evaluation target recorded data XN from theunknown data group C2, the calculating unit 42 performs the processing(SA1) of calculating the N time differences O12 to ON·1 calculated forthe N recorded data pairs P12 to PN·1, respectively, and the evaluatingunit 44 performs the processing (SB1 to SB5) of evaluating whether ornot the N pieces of recorded data X1 to XN include the recorded data XNhaving no relation. The recorded data analyzing unit 40 (the selectingunit 48, the calculating unit 42, and the evaluating unit 44) in thesecond embodiment can be said to perform processing of excluding therecorded data XN having no relation to the related data group C1 fromthe unknown data group C2.

The second embodiment also achieves effects similar to those of thefirst embodiment. The second embodiment can particularly classify aplurality of pieces of recorded data X unknown as to presence or absenceof relation thereof into the related data group C1 and the unrelateddata group C3. In addition, in the second embodiment, each time theevaluation target recorded data XN is moved to the related data groupC1, the recorded data X in the unrelated data group C3 is moved to theunknown data group C2. Hence, even recorded data X once moved to theunrelated data group C3 can be sorted into the related data group C1anew when the recorded data X has relation to recorded data X newlymoved to the related data group C1. That is, all of a plurality ofpieces of recorded data X having relation can be sorted into the relateddata group C1.

<Modifications>

Each of the embodiments illustrated above can be modified in variousmanners. Modifications will be illustrated in the following. Two or moremodifications arbitrarily selected from the following illustrations canbe integrated with each other as appropriate within a scope where nomutual inconsistency arises.

(1) In each of the foregoing embodiments, the time difference Oij iscalculated according to the time difference τ between the audio signalsy(t) included in the respective pieces of recorded data Xi and Xj of therecorded data pair Pij. However, the signals used for the calculation ofthe time difference τ are not limited to the audio signals y(t). Forexample, when the audio of each of the pieces of recorded data Xincludes common utterance contents, the time difference Oij can also becalculated by analyzing the utterance contents of the respective piecesof recorded data Xi and Xj by voice recognition, and comparing a resultof the analysis between the two pieces of recorded data Xi and Xj. Inaddition, the time difference Oij may be calculated by comparing (forexample, calculating the cross-correlation Cij(τ)), between the twopieces of recorded data Xi and Xj, signals indicating temporal changesin a feature quantity (for example, pitch) extracted from the audiosignals y(t). Further, the time difference Oij can also be calculated bygenerating signals indicating temporal changes in lightness of imagesincluded in the recorded data pair Pij, for example, from moving imagesignals representing temporal changes in the images (that is, movingimages), and comparing the signals between the two pieces of recordeddata Xi and Xj. That is, the time difference τ can also be calculated byusing signals of some kind which signals are generated from signals(audio signals and moving image signals) representing temporal changesof the recording target. As is understood from the above description,the signals used to calculate the time difference Oij arecomprehensively expressed as time signals representing temporal changesof the recording target (audio or a moving image) in the respective twopieces of recorded data Xi and Xj of the recorded data pair Pij. Thatis, the concept of the time signals includes not only signals (audiosignals and moving image signals) representing temporal changes of therecording target itself but also signals representing temporal changesin feature quantities of the recording target (signals indirectlyrepresenting temporal changes of the recording target). However, inconsideration of a tendency for the audio signals y(t) to have smalldifferences in temporal variation according to a recording condition(for example, photographing positions), the configuration of each of theforegoing embodiments using the audio signals y(t) has an advantage ofbeing able to identify the time differences O12 to ON·1 between the Npieces of recorded data X1 to XN with high accuracy as compared with aconfiguration using time signals such as moving images or the like.

(2) In each of the foregoing embodiments, the time difference Oij of therecorded data pair Pij is calculated according to the cross-correlationCij(τ). However, the index used for the calculation of the timedifference Oij is not limited to the cross-correlation Cij(τ). Forexample, the time difference Oij of the recorded data pair Pij can alsobe calculated according to a normalized cross-correlation. The indexused for the calculation of the time difference Oij is arbitrary as longas the time difference Oij can be calculated between time signalsrepresenting temporal changes of the recording target in the tworespective pieces of recorded data Xi and Xj of the recorded data pairPij.

(3) In each of the foregoing embodiments, the time difference τmaximizing the absolute value |Cij(τ)| of the cross-correlation Cij(τ)is determined as the time difference Oij. However, the time differenceOij can also be selected from a plurality of candidate values identifiedfrom the absolute value |Cij(τ)| of the cross-correlation Cij(τ). Forexample, the calculating unit 42 identifies a plurality of timedifferences τ corresponding to mutually different maxima of the absolutevalue |Cij(τ)| of the cross-correlation Cij(τ) (for example, maxima of amoving average of the absolute value |Cij(τ)|) as candidates for thetime difference Oij, and calculates one of the plurality of candidatevalues as the time difference Oij. For example, a candidate valueminimizing the absolute value of the total value S among the pluralityof candidate values is selected as the time difference Oij. The aboveconfiguration can identify the time difference Oij with high accuracy.

(4) In each of the foregoing embodiments, the evaluating unit 44evaluates whether or not the recorded data XN having no relation isincluded in the N pieces of recorded data X1 to XN including the N−1pieces of recorded data X1 to XN−1 having relation to each other and onepiece of unknown recorded data XN having unknown relation. However, thepresence or absence of relation can also be evaluated according to thetotal value S of the N time differences O12 to ON·1 for the N pieces ofrecorded data X1 to XN including a plurality of pieces of recorded dataX having relation to each other and a plurality of pieces of unknownrecorded data X. That is, there may be a plurality of pieces of unknownrecorded data X.

(5) In each of the foregoing embodiments, the edit processing unit 46 isincorporated in the recorded data editing system 10. However, the editprocessing unit 46 can also be incorporated in a server device or aterminal device separate from the recorded data editing system 10. Inthis case, the recorded data editing system 10 transmits a plurality ofpieces of recorded data X related to each other to the server device orthe terminal device. As is understood from the above description, therecorded data editing system 10 in each of the foregoing embodiments isan illustration of a device (that is, a recorded data processing device)including the recorded data analyzing unit 40 that excludes the recordeddata XN having no relation to the N−1 pieces of recorded data X1 to XN−1having relation to each other by analyzing the time differences O12 toON·1 for the N pieces of recorded data X1 to XN. Edit processing (theedit processing unit 46) is not essential in the recorded dataprocessing device according to the some embodiments.

(6) In the second embodiment, the presence or absence of relation of allof the plurality of pieces of unknown recorded data X stored by thestorage device 24 is evaluated. However, it is also possible to evaluatethe presence or absence of relation of a part of the unknown recordeddata X stored by the storage device 24. For example, the recorded dataanalyzing unit 40 obtains time information indicating a recording period(for example, a start time and an end time) of each piece of recordeddata X from the storage device 24 together with the unknown recordeddata X, identifies the unknown recorded data X whose recording timeindicated by the time information overlaps those of the N−1 pieces ofrecorded data X on the time axis, and performs operation similar to thatof the second embodiment. That is, the unknown recorded data X thatclearly has no relation to the N−1 pieces of recorded data X is excludedfrom processing targets. Incidentally, while the presence or absence ofrelation between the unknown recorded data X and the N−1 pieces ofrecorded data X can be evaluated from the time information, timesmeasured by the recording devices can actually have errors in therespective recording devices. There is thus a meaning in evaluating thepresence or absence of relation of the unknown recorded data X by theconfiguration of the second embodiment. The above configuration canexclude the unknown recorded data X having no relation to the N−1 piecesof recorded data X from processing targets, and therefore reduce aprocessing load on the recorded data analyzing unit 40 as compared withthe configuration that sets all of the unknown recorded data X stored bythe storage device 24 as processing targets.

(7) The recorded data analyzing unit 40 illustrated in each of theforegoing embodiments is implemented by cooperation between theelectronic controller 22 and a program, as described above. The programcan be provided in a form of being stored on a recording medium readableby a computer, and installed on the computer. The recording medium is,for example, a non-transitory recording medium. An optical recordingmedium (optical disk) such as a compact disc read-only memory (CD-ROM)or the like is a good example of the recording medium. However, therecording medium can include publicly known arbitrary types of recordingmedia such as a semiconductor recording medium, a magnetic recordingmedium, and the like. It is to be noted that the non-transitoryrecording medium includes arbitrary recording media excluding transitorypropagating signals, and does not exclude volatile recording media. Inaddition, the program can also be provided to a computer in a form ofdistribution via a communication network.

(8) In some embodiments can also be identified as an operating method(recorded data processing method) of the recorded data analyzing unit 40according to each of the foregoing embodiments. Specifically, in arecorded data processing method according to a first mode of the presentinvention, for each of N (N is a natural number of three or more)recorded data pairs P12 to PN·1 each formed by two pieces of recordeddata Xi and Xj that are adjacent to each other when N pieces of recordeddata X each representing a recording target including at least one ofaudio or video are cyclically arranged, a time difference Oij iscalculated between time signals representing temporal changes of therecording target in the two respective pieces of recorded data Xi and Xjof the recorded data pair Pij, and whether or not the N pieces ofrecorded data X1 to XN include recorded data XN having no relation isevaluated according to a total value S of time differences Oijcalculated for the N recorded data pairs P12 to PN·1, individually.

In addition, in a recorded data processing method according to a secondmode of the present invention, evaluation target recorded data XN issequentially selected from an unknown data group C2 including aplurality of pieces of recorded data X each representing a recordingtarget including at least one of audio or an image, at each time ofselection of the evaluation target recorded data XN, for each of N (N isa natural number of three or more) recorded data pairs P12 to PN·1 eachformed by two pieces of recorded data Xi and Xj that are adjacent toeach other when N pieces of recorded data X are cyclically arranged, theN pieces of recorded data X including a related data group C1 includinga plurality of pieces of recorded data X each representing the recordingtarget including the at least one of the audio or the image and havingrelation to each other and the evaluation target recorded data XN, atime difference Oij is calculated between time signals representingtemporal changes of the recording target in the two respective pieces ofrecorded data Xi and Xj of the recorded data pair Pij, and presence orabsence of relation of the evaluation target recorded data XN to therelated data group C1 is evaluated according to a total value S of timedifferences Oij calculated for the N recorded data pairs P12 to PN·1,individually, and when the relation is affirmed, the evaluation targetrecorded data XN is moved from the unknown data group C2 to the relateddata group C1.

(9) The following constitutions, for example, are grasped from theembodiments illustrated above.

A recorded data processing method according to one aspect includes:calculating, for each of N (N is a natural number of three or more)recorded data pairs each formed by two pieces of recorded data that areadjacent to each other when N pieces of recorded data each representinga recording target including at least one of audio or an video arearranged cyclically, a time difference between time signals representingtemporal changes of the recording target in the two respective pieces ofrecorded data of the recorded data pair; and evaluating whether or notthe N pieces of recorded data include recorded data having no relationaccording to a total value of N time differences calculated for the Nrecorded data pairs, individually. The above method can appropriatelyevaluate whether or not the N pieces of recorded data include recordeddata having no relation according to the total value of the N timedifferences individually calculated for the N recorded data pairs whenthe N pieces of recorded data each representing the recording targetincluding the at least one of the audio or video are arrangedcyclically.

In the recorded data processing method according to another aspect, inthe calculating of the time difference, the time difference iscalculated for each recorded data pair of the N pieces of recorded dataincluding N−1 pieces of recorded data having relation to each other andone piece of recorded data having unknown relation to the N−1 pieces ofrecorded data, and in the evaluating, when the N pieces of recorded dataare evaluated as including recorded data having no relation, the onepiece of recorded data is decided to be recorded data having no relationto the N−1 pieces of recorded data. In the above method, when the Npieces of recorded data are evaluated as including recorded data havingno relation, the recorded data having no relation to the N−1 pieces ofrecorded data is identified. Merely evaluating the presence or absenceof recorded data having no relation according to a total value of timedifferences of N pieces of recorded data that are unknown as to whetheror not there is relation there between cannot identify recorded datahaving no relation to the other recorded data among the N pieces ofrecorded data. The above-described preferred method evaluates thepresence or absence of relation according to the total value of the timedifferences of the respective recorded data pairs of the N pieces ofrecorded data including the N−1 pieces of recorded data already known tohave relation to each other and one piece of recorded data havingunknown relation to the N−1 pieces of recorded data. It is thereforepossible to determine whether or not the one piece of recorded data hasrelation to the N−1 pieces of recorded data.

In the recorded data processing method according to another aspect,content in which a plurality of pieces of recorded data having therelation among the N pieces of recorded data are synchronized with eachother are generated. The above method can generate the content in whichthe plurality of pieces of recorded data having relation to each otheramong the N pieces of recorded data are synchronized with each other.

A recorded data processing method according to another aspect includes:sequentially selecting evaluation target recorded data from an unknowndata group including a plurality of pieces of recorded data eachrepresenting a recording target including at least one of audio orvideo; at each time of selection of the evaluation target recorded data,for each of N (N is a natural number of three or more) recorded datapairs each formed by two pieces of recorded data that are adjacent toeach other when N pieces of recorded data are arranged cyclically, the Npieces of recorded data including a related data group including aplurality of pieces of recorded data each representing the recordingtarget including the at least one of the audio or the image and havingrelation to each other and the evaluation target recorded data,calculating a time difference between time signals representing temporalchanges of the recording target in the two respective pieces of recordeddata of the recorded data pair; and evaluating presence or absence ofrelation of the evaluation target recorded data to the related datagroup according to a total value of N time differences calculated forthe N recorded data pairs, individually, and moving the evaluationtarget recorded data from the unknown data group to the related datagroup when the relation is affirmed. The above method evaluates, foreach (evaluation target recorded data) of the plurality of pieces ofrecorded data unknown as to the presence or absence of the relation, thepresence or absence of relation thereof to the plurality of pieces ofrecorded data already known to have relation to each other, and adds theevaluation target recorded data evaluated as having the relation to therelated data group. Hence, it is possible to classify the plurality ofpieces of recorded data unknown as to the presence or absence ofrelation thereof into a set of the plurality of pieces of recorded dataalready known to have relation to each other (related data group) and aset of recorded data having no relation.

In the recorded data processing method according to another aspect,content in which the plurality of pieces of recorded data included inthe related data group are synchronized with each other are generated.The above method can generate the content in which the plurality ofpieces of recorded data having relation to each other are synchronizedwith each other.

In the recorded data processing method according to another aspect, inthe evaluating of the presence or absence of the relation, recorded datawhose relation to the related data group is negated is moved from theunknown data group to an unrelated data group, and each time theevaluation target recorded data is moved to the related data group,recorded data included in the unrelated data group is moved to theunknown data group. The above method moves the recorded data in theunrelated data group to the unknown data group each time the evaluationtarget recorded data is moved to the related data group. Thus, evenrecorded data once moved to the unrelated data group can be included inthe related data group anew when the recorded data has relation torecorded data newly moved to the related data group. Hence, all of aplurality of pieces of recorded data having relation to each other canbe sorted into the related data group.

In the recorded data processing method according to another aspect, thetime signals are audio signals representing temporal changes in theaudio. The above method calculates a total value of time differencesbetween audio signals in the N recorded data pairs, individually. Hence,because time signals of movie or the like have large differences intemporal variation according to a recording condition (for example,capturing positions), but the audio signals have small differences intemporal variation according to the recording condition, it is possibleto identify time differences between a plurality of pieces of recordeddata with high accuracy, and more appropriately evaluate whether or notrecorded data having no relation is included.

In the recorded data processing method according to another aspect, inthe calculating of the time difference, a plurality of candidate valuesfor the time difference between the time signals representing thetemporal changes of the recording target in the two respective pieces ofrecorded data of the N recorded data pairs are identified, and one ofthe plurality of candidate values is identified as the time difference.The above method can identify the time difference with high accuracy ascompared with the method of adopting the sole time difference identifiedfrom between the two pieces of recorded data as a final value.

A recorded data processing device according to another aspect includesan electronic controller that has a calculating unit and an evaluatingunit, wherein the calculating unit is configured to calculate, for eachof N (N is a natural number of three or more) recorded data pairs eachformed by two pieces of recorded data that are adjacent to each otherwhen N pieces of recorded data each representing a recording targetincluding at least one of audio or video are arranged cyclically, a timedifference between time signals representing temporal changes of therecording target in the two respective pieces of recorded data of therecorded data pair; and the evaluating unit is configured to evaluatewhether or not the N pieces of recorded data include recorded datahaving no relation according to a total value of N time differencescalculated for the N recorded data pairs, individually. The aboveconfiguration can appropriately evaluate whether or not the N pieces ofrecorded data include recorded data having no relation according to thetotal value of the N time differences individually calculated for the Nrecorded data pairs when the N pieces of recorded data each representingthe recording target including the at least one of the audio or theimage are arranged cyclically.

A recorded data processing device according to another aspect includesan electronic controller that has a selecting unit, a calculating unitand an evaluating unit, wherein the selecting unit is configured tosequentially select evaluation target recorded data from an unknown datagroup including a plurality of pieces of recorded data each representinga recording target including at least one of audio or video; acalculating unit is configured to calculate, at each time of selectionof the evaluation target recorded data, for each of N (N is a naturalnumber of three or more) recorded data pairs each formed by two piecesof recorded data that are adjacent to each other when N pieces ofrecorded data are arranged cyclically, the N pieces of recorded dataincluding a related data group including a plurality of pieces ofrecorded data each representing the recording target including the atleast one of the audio or the image and having relation to each otherand the evaluation target recorded data, a time difference between timesignals representing temporal changes of the recording target in the tworespective pieces of recorded data of the recorded data pair; and theevaluating unit is configured to evaluate, at each time of selection ofthe evaluation target recorded data, presence or absence of relation ofthe evaluation target recorded data to the related data group accordingto a total value of N time differences calculated for the N recordeddata pairs, individually, and add the evaluation target recorded data tothe related data group when the relation is affirmed. According to theabove configuration, it is possible to classify the plurality of piecesof recorded data unknown as to the presence or absence of relationthereof into a set of the plurality of pieces of recorded data alreadyknown to have relation to each other (related data group) and a set ofrecorded data having no relation.

The invention claimed is:
 1. A recorded data processing method,comprising: calculating a time difference between time signalsrepresenting temporal changes of a recording target in two pieces ofrecorded data of each recorded data pair of N recorded data pairs,wherein N is greater than or equal to 3, each recorded data pair of theN recorded data pairs is formed by the two pieces of the recorded datathat are adjacent to each other, the two pieces of the recorded databelongs to N pieces of the recorded data arranged cyclically, and the Npieces of the recorded data each represents the recording targetincluding at least one of audio or video; and evaluating whether or notthe recorded data includes one piece of the recorded data having norelevance in terms of one of content or time to other pieces of therecorded data among the N pieces of the recorded data, according to atotal value of N time differences calculated for the N recorded datapairs, wherein the time difference is calculated for each recorded datapair of the N pieces of the recorded data including N−1 pieces of therecorded data having relation to each other and the one piece of therecorded data having unknown relation to the N−1 pieces of the recordeddata, and when the recorded data is evaluated as including the one pieceof the recorded data having no relevance among the N pieces of therecorded data, the one piece of the recorded data is determined to berecorded data having no relation to the N−1 pieces of the recorded data.2. The recorded data processing method according to claim 1, whereincontent in which a plurality of pieces of the recorded data having therelation among the N pieces of the recorded data are synchronized witheach other is generated.
 3. The recorded data processing methodaccording to claim 1, wherein the time signals are audio signalsrepresenting the temporal changes in the audio.
 4. The recorded dataprocessing method according to claim 1, wherein in the calculating ofthe time difference, a plurality of candidate values for the timedifference between the time signals representing the temporal changes ofthe recording target in the two pieces of the recorded data of the Nrecorded data pairs are respectively identified, and one of theplurality of candidate values is identified as the time difference.
 5. Arecorded data processing method, comprising: sequentially selectingevaluation target recorded data from an unknown data group including aplurality of pieces of recorded data each representing a recordingtarget including at least one of audio or an image; at each time ofselection of the evaluation target recorded data, for each of N recordeddata pairs each formed by two pieces of recorded data that are adjacentto each other when N pieces of recorded data are arranged cyclically,the N pieces of recorded data including a related data group includingthe plurality of pieces of recorded data each representing the recordingtarget including the at least one of the audio or the image and havingrelation to each other and the evaluation target recorded data,calculating a time difference between time signals representing temporalchanges of the recording target in the two pieces of recorded data ofeach recorded data pair, wherein N is a natural number of three or more;evaluating presence or absence of relation of the evaluation targetrecorded data to the related data group according to a total value of Ntime differences calculated for the N recorded data pairs, individually;and moving the evaluation target recorded data from the unknown datagroup to the related data group when the relation is affirmed.
 6. Therecorded data processing method according to claim 5, wherein content inwhich the plurality of pieces of recorded data included in the relateddata group are synchronized with each other is generated.
 7. Therecorded data processing method according to claim 5, wherein, in theevaluating of the presence or absence of the relation, first recordeddata whose relation to the related data group is negated is moved fromthe unknown data group to an unrelated data group, and each time theevaluation target recorded data is moved to the related data group,second recorded data included in the unrelated data group is moved tothe unknown data group.
 8. The recorded data processing method accordingto claim 5, wherein the time signals are audio signals representing thetemporal changes in the audio.
 9. The recorded data processing methodaccording to claim 5, wherein in the calculating of the time difference,a plurality of candidate values for the time difference between the timesignals representing the temporal changes of the recording target in thetwo pieces of recorded data of the N recorded data pairs arerespectively identified, and one of the plurality of candidate values isidentified as the time difference.
 10. A recorded data processingdevice, comprising: an electronic controller configured to: calculate atime difference between time signals representing temporal changes of arecording target in two pieces of recorded data of each recorded datapair of N recorded data pairs, wherein N is greater than or equal to 3,each recorded data pair of the N recorded data pairs is formed by thetwo pieces of the recorded data that are adjacent to each other, the twopieces of the recorded data belongs to N pieces of the recorded dataarranged cyclically, and the N pieces of the recorded data eachrepresents the recording target including at least one of audio or animage; and evaluate whether or not the recorded data includes one pieceof recorded data having no relevance in terms of one of content or timeto other pieces of the recorded data among the N pieces of the recordeddata, according to a total value of N time differences calculated forthe N recorded data pairs, wherein the time difference is calculated foreach recorded data pair of the N pieces of the recorded data includingN−1 pieces of the recorded data having relation to each other and theone piece of the recorded data having unknown relation to the N−1 piecesof the recorded data, and when the recorded data is evaluated asincluding the one piece of the recorded data having no relevance amongthe N pieces of the recorded data, the one piece of the recorded data isdetermined to be recorded data having no relation to the N−1 pieces ofthe recorded data.
 11. A recorded data processing device, comprising: anelectronic controller configured to: sequentially select evaluationtarget recorded data from an unknown data group including a plurality ofpieces of recorded data each representing a recording target includingat least one of audio or an image; calculate, at each time of selectionof the evaluation target recorded data, for each of N recorded datapairs each formed by two pieces of recorded data that are adjacent toeach other when N pieces of recorded data are arranged cyclically, the Npieces of recorded data including a related data group including theplurality of pieces of recorded data each representing the recordingtarget including the at least one of the audio or the image and havingrelation to each other and the evaluation target recorded data, a timedifference between time signals representing temporal changes of therecording target in the two pieces of recorded data of each recordeddata pair, wherein N is a natural number of three or more; evaluate, ateach time of the selection of the evaluation target recorded data,presence or absence of relation of the evaluation target recorded datato the related data group according to a total value of N timedifferences calculated for the N recorded data pairs; and add theevaluation target recorded data to the related data group when therelation is affirmed.